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IMAGING SYSTEM AND METHOD FOR 
INTERACTIVE CONTROL OF IMAGE QUALITY 



5 BACKGROUND OF THE PRESENT INVENTION 

Field of the Invention 

The present invention relates generally to an electronic imaging system and 
method, particularly, to an improved imaging system and method for customizing the 
images to the viewer's specifications, and, more particularly, to an imaging system and 
10 method allowing the viewer, via a backchannel, to adjust the spatial and temporal 
resolution and quantization parameters of an image. 

Background and Objects of the Present Invention 

With the rise of the consumer electronics industry over the past few 
decades, a variety of electronic imaging systems of increasing complexity have 
15 emerged, e. g. , video recorders, camcorders and the like. Additionally, video 

teleconferencing communications are becoming increasingly important as our 
society becomes increasingly and interactively interconnected. 

As is understood in this art, video, Le,, moving, images undergo 
encoding to reduce the amount of information needed to represent a given 
20 image. Encoding affects both the spatial resolution, Le^, the detail within a 

particular image frame, and temporal resolution, Le^, the number of such image 
frames per second. These parameters are typically fixed within a conventional 
video system, such as the one shown in FIGURE 1 of the Drawings and 
generally referred to herein as numeral 10. The video system 10 in the figure 
25 includes a sending device 12 which receives signals from a camera 14. It should 

be understood that various portions of camera 14 which are not related to the 
present invention, for example, the diaphragm, shutter and the like, are not 
illustrated. Accordingly, as is understood in this art, the optical image before 
the camera 14, such as the individual depicted, is received by a camera lens 16 
30 and converted into an analog video signal, e^, by a conventional charge 

coupled device. It should be understood that camera 14 may be a digital camera 
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forwarding digital data to a subsampler device 1 8 within the sending device 12. 
If camera 14 is not digital, however, and analog-to-digital conversion is 
required, then device 18 may also function as an A/D converter, as is 
understood in the art. The subsampler 1 8 determines pixel values representing 
5 the captured video image at a particular spatial resolution, Le., pixels per line 

and lines per image, and temporal resolution, Le^, images per second. Another 
parameter related to both spatial and temporal resolution is quantization, ie,, a 
measure of the amount of distortion present in the video signal, as will be 
discussed in more detail hereinafter. 
10 An encoder 20 encodes the aforedescribed digital image data into a 

video signal stream which flows into a buffer 22. As is understood in the art and 
discussed further herein, the rate of the flow of information from the encoder 20 
into buffer 22 varies in accordance with the degree of encoding. Additionally, 
the video signal stream typically includes compressed signals, in which image 
15 information has been condensed or compressed by the encoder 20 to facilitate 

transmission or storage. One set of formats using such compression 
technologies are those specified by the Moving Picture Experts Group (MPEG), 
a standard in accord with the International Organization for Standardization/ 
International Electro-technical Commission (ISO/IEC). Other compression 
20 technologies are the H.261, H.262 and H.263 standards of the International 

Telecommunications Union, Teleconferencing Section (ITU-T) for use in video 
teleconferencing, for example. 

In conjunction with these image data formatting standards and 
techniques, by which the encoder 20 provides a syntax for the subsequent 
25 bitstream, the encoder 20 also employs compression algorithms, such as 

Discrete Cosine Transforms (DCT), Huffman coding and other mechanisms, 
whereby the amount of data needed to represent the image is drastically reduced 
while substantially retaining image integrity. As is well understood by those 
skilled in the art, these and other techniques eliminate or reduce the transmission 
30 of frame-to-frame redundancies and other information which are unnecessary or 
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repetitive, and exploit various physiological and psychological aspects of human 
perception to present a coherent image to the viewer's eye. 

With further reference to FIGURE 1, the subsampler 18, encoder 20 
and buffer 22 are controlled by a control unit 24, which also controls other 
5 functions of the imaging system 10. For example, control unit 24 controls the 

sequencing of the afore-described operations, Le,, image pickup by camera 14 
through a connection thereto (not shown), pixel conversion in subsampler 18, 
compression in encoder 20, recording the encoded images on a magnetic or 
electronic recording medium (not shown), and other operations. Control unit 

10 24 supplies encoder 20 with a plurality of operating parameters to govern the 

aforementioned transformation of pixel data into a corresponding compressed 
bitstream. As discussed, control unit 24 also governs the variable bit rate of the 
information flow into buffer 22 to maintain a particular data level and avoid both 
overflow and underflow therein. 

15 As is understood in this art, the primary purpose of buffer 22 is to 

regulate the flow of data from the encoder 20 and forward that data at a fixed 
rate across a transmission channel 26 to a receiver device 28, particularly, to 
another buffer 30 therein, which like buffer 22 acts as a reservoir storing the 
data and regulating its use. It should, of course, be understood that channel 26 

20 may transfer data at a variable rate, £4^, a variable rate service of an 

Asynchronous Transfer Mode (ATM) network. Nonetheless, the variable flow 
rate of data from encoder 20 does not generally agree with that of channel 26, 
fixed or variable. 

Buffer 30 forwards the received image data, at a fixed or variable rate 
25 as needed, to a decoder 32. Similarly to the encoding process, the decoder 32 

reverses the aforedescribed compression algorithms to expand the image 
pursuant to the aforementioned operating parameters. In other words, the 
decoder 32 decompresses the compressed information in the bit stream and 
reconstitutes the image pursuant to the relevant image format used, e. g. . the 
30 ITU-R/601 Digital Studio Standard, and the operating parameters. The 

reconstituted image is then placed within an image storage device 34, the 
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contents of which may be continuously displayed on a video display 36, the 
circuitry of which is understood in the art. 

As discussed, the aforedescribed compression technologies employ 
various techniques to condense the image information. The decoder 32 is 
5 configured to interpret the format and operating parameters by which the image 

information was encoded by encoder 20. As is understood in the art, much of 
the decoding process performed within the decoder 32 may be called 
"normative", Le,, fixed by the particular standard used, e.g. . MPEG. 
Consequently, the decoder 32 readily recognizes these normative parts of a 
10 signal from encoder 20, L^, how to interpret the transmitted bits in the bit 

stream. 

In conventional apparatus employing the above technology, the 
aforementioned operating parameters are fixed within the video system 10. 
Usually, encoder 20 utilizes fixed spatial and temporal resolution values, which 

1 5 comports well with the requirements of buffer 22, guaranteeing a fixed-rate 

bitstream across transmission channel 26. Nonetheless, buffer 22 in an effort to 
maintain the transmission rate required by the channel 26 adjusts the 
quantization or distortion of the pertinent images. Quantization then becomes 
a function of the fullness of buffer 22, which, in turn, is a function of the 

20 complexity of the subject video images, Le,, how bit-consuming the images are 

during compression. Some encoders 20 have fixed spatial resolution only and 
the buffer 22 adjusts quantization and temporal resolution to maintain the 
constant bit-rate. The balance between quantization and temporal resolution is 
governed by a buffer regulation algorithm, as is understood in the art. 

25 One problem with the above configuration, however, is that the 

aforementioned operating parameters may be unsuitable in certain 
circumstances, and the buffer regulation algorithm or other resolution balancing 
scheme may require adjustment to suit the needs of the human viewer who may 
have a different spatial/temporal resolution and distortion balance in mind. For 

30 example, some video applications may require higher temporal resolution at the 

cost of coarse quantization, £*g., video communication between deaf people 
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(sign language) who prefer high temporal resolution. Additionally, surveillance 
applications normally require higher spatial resolution and fine quantization at 
the cost of temporal resolution. 

Further, with the growing rise of consumer use and proficiency in 
electronic imaging systems, sophisticated videographers want increasing control 
over the operating parameters and may make fine adjustments to the balance 
between spatial and temporal resolution and quantization for a multitude of 
other applications, fine tuning these parameters for objective or subjective 
effect. 

With these operating parameters fixed, however, videographers or any 
other user of video apparatus having these immutable characteristics cannot 
make any adjustments to the apparatus and encoder 20 operates without any 
feedback from the viewer. 

Accordingly, it is a first object of the present invention to provide the 
viewer with a means to adjust the aforementioned spatial and temporal 
resolutions and quantization variables to suit their individual needs. 

It is more particular object of the present invention to provide a means 
of feedback from the viewer to the encoder, enabling the viewer to have 
increased flexibility over the aforementioned operating parameters. 
20 

SUMMARY OF THE INVENTION 

The present invention is directed to an improvement in an imaging 
system and method for providing increased viewer control over the operating 
parameters of the imaging system. Viewer control is facilitated by including a 

25 backchannel within the imaging system, enabling the viewer to adjust the 

operating parameters, e.g. , spatial and temporal resolution, and quantization, of 
the video transmission. 

A more complete appreciation of the present invention and the scope 
thereof can be obtained from the accompanying drawings which are briefly 

30 summarized below, the following detailed description of the presently-preferred 

embodiments of the invention, and the appended claims. 



10 
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BRIEF DESCRIPTION OF THE DRAWINGS 

FIGURE 1 is a block diagram illustrating a conventional electronic 
imaging system; and 

FIGURE 2 is a block diagram illustrating an electronic imaging system 
5 in accordance with the present invention. 

DETAILED DESCRIPTION OF THE PRESENTLY PREFERRED 
EXEMPLARY EMBODIMENTS 

The present invention will now be described more fully hereinafter 

10 with reference to the accompanying drawings, in which preferred embodiments 

of the invention are shown. This invention may, however, be embodied in many 
different forms and should not be construed as limited to the embodiments set 
forth herein; rather, these embodiments are provided so that this disclosure will 
be thorough and complete, and will fully convey the scope of the invention to 

1 5 those skilled in the art. 

With reference now to FIGURE 2 of the drawings, there is illustrated 
an electronic imaging system, generally represented by the numeral 40, which 
incorporates the subject matter of the present invention. The imaging system 
40 illustrated in FIGURE 2, as with the conventional imaging system 10 set 

20 forth in FIGURE 1, includes a sending device 42 which receives signals from a 

camera 44 which captures and records an optical image, such as the individual 
depicted. As discussed, the various portions of camera 44 that are not related 
to the present invention, are not illustrated. The optical image before camera 
44 is received by a camera lens 46 and converted into a signal, as described. 

25 The aforementioned signal is then forwarded to a subsampler device 48, which 

determines pixel values representing the captured video image at a particular 
spatial resolution, L^, pixels per line and lines per image, and temporal 
resolution, Le,, images per second. As discussed in connection with the imaging 
system 10 of FIGURE 1, an encoder 50 encodes the aforedescribed digital data 

30 into a video signal stream at particular spatial and temporal resolutions, which 

are fixed in conventional systems. 
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As with the sending device 12 of the conventional imaging system 10, 
sending device 42 includes a buffer 52 to receive the encoded, compressed video 
signal stream from the encoder 50, and a control unit 54 to control the 
operations of the converter 48, encoder 50 and buffer 52. As discussed, the 
5 control unit 54 supplies encoder 50 with the aforedescribed operating 

parameters to govern the data transformation. Whereas several parameters were 
fixed in the conventional system illustrated in FIGURE 1, e.g. . the 
aforementioned spatial and temporal resolutions, a user of the video system 40 
of the present invention is able to variably control these parameters, as will be 

10 discussed more fully hereinafter. 

As with the conventional system, buffer 52 manages the variable flow 
of data from the encoder 50 and outputs a fixed-rate video data bit stream 
across a transmission channel 56 to a receiver device 58, particularly, to a buffer 
60 therein, which like buffer 30 in FIGURE 1 receives the fixed flow of data and 

1 5 forwards it to a decoder 62. As with channel 26, it should be understood that 

channel 56 may permit variable data flow rates. 

Similarly to the encoding process, the decoder 62 reverses the 
aforedescribed compression algorithms to expand the image pursuant to the 
aforementioned operating parameters. Decoder 62 decompresses the 

20 compressed and formatted information in the bit stream and reconstitutes the 

image pursuant to the relevant format and the operating parameters. The 
reconstituted image is then placed within an image storage device 64, the 
contents of which may be continuously displayed on a video display 66, which 
in FIGURE 2 displays the aforementioned individual. 

25 As discussed, decoder 62 is configured to decode normative 

information from the encoder 50, Le,, the format, . standard and operating 
parameters of the video data bitstream. It should, therefore, be understood that 
the decoder 62, as well as decoder 32, recognizes these normative parts of a 
signal from encoder 50, e.g. . the particular video format used, e.g. t the 

30 aforementioned ITU-R/601 standard, and the various compression standards, 

e.g.. ISO/EC MPEG-1, MPEG-2, and MPEG-4, and ITU-T H.261, H.262 and 
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H.263. Although decoder 62 is preferably of conventional design and therefore 
able to understand the pertinent normative communication signals, it should be 
understood that decoder 62 may also be configured to accept non-normative 
commands, L^, commands or information outside the particular standard being 
5 used, as described hereinafter. 

Since the encoder 50 (and the encoding process) is not specified within 
the aforementioned standards (all that matters is correct decoding), video system 
designers have a lot of freedom as to the implementation non-normative aspects 
of the technology. So long as the decoder 62 can understand the pertinent 

10 standards and is configured to decode the particular non-normative functions 

desired, numerous additional options may be implemented, such as in modifying 
picture quality, as will be discussed more fully hereinafter. 

With reference again to FIGURE 2, receiving device 58 also includes 
a human interface device 68, through which many of the aforedescribed 

15 operating parameters may be adjusted, e.g. . to modify image clarity (spatial 

resolution and quantization), frequency (temporal resolution or frame rate) and 
other characteristics. The human interface device 68, which may include a 
button, slide, keyboard or other conventional interface apparatus, forwards the 
indicated changes to a translator 70, which converts the changes to a signal. 

20 The aforedescribed signal is then sent back to the control unit 54 of the sending 

device 42 via a backchannel 72. 

By means of the video system 40 configuration with backchannel 72 
signaling capability, the viewer may through interface 68 interactively modify 
various operational parameters, such as those defining image quality. For 

25 example, the following operational parameters may be implemented in the video 

system 40 of the present invention to regulate image quality: 

(a) high/low spatial resolution (image detail), 

(b) high/low temporal resolution (frame rate), 

(c) high/low quantization (image distortion), 
30 (d) balance between (a) and (b), 

(e) balance between (a) and (c), and 
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(f) balance between (b) and (c). 
It should be understood that, whereas the conventional video system 1 0 employs 
fixed (a), (b) and (c) operational parameters, the video system 40 of the present 
invention permits alteration of the balance between these operational 
5 parameters. In the conventional video system 10, if either (a) or (b) were 

changed, then the amount of distortion (c) must be adjusted to achieve the 
requisite bit rate. Accordingly, selecting (a) is equivalent to selecting (e), a 
balance between spatial resolution and quantization. Similarly, selecting (b) is 
equivalent to selecting (f), a balance between temporal resolution and 

10 quantization. It should also be understood that selecting (c) is also equivalent 

to selecting (f) since an encoder typically does not alter its spatial resolution; but 
may easily adjust temporal resolution. As discussed, the viewer may want a 
different image resolution balance than that presently in use, L^, either the 
aforementioned predetermined balance or a previously selected balance, and 

1 5 want to adjust the operational parameter settings to achieve a desired balance. 

Through interaction with the human interface 68, e.g. . by pressing or turning a 
button 74 (constituting interface 68 or connected thereto via a connection 75) 
on the display device 74 or a like button 76 on a remote device 78 also shown 
in FIGURE 2, the translator 70 may forward a particular codeword or other 

20 indicia indicating the particular command corresponding thereto back to the 

encoder 50, which adjusts its operations accordingly. For example, the viewer 
may forward the commands "A+" to increase spatial resolution, "B-" to 
decrease temporal resolution, ct F+" to decrease distortion, etc. It should, of 
course, be understood that the above commands are exemplary only, and other 

25 symbols may be utilized by the video system 40 to implement the viewer's 

desired changes. In any event, the bitstream after adjustment should reflect the 
indicated change. 

Additionally, a request for a new balance between temporal resolution 
and coding distortion may be sent from translator 70 via backchannel 72 as 

30 "q<k>", where "q" represents the type of command requested and "<k> M 

indicates a particular value for that command. It should be understood that "k" 
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is preferably within a predefined discrete range of permissible values. Since the 
number of permissible resolutions is rather large in the aforementioned 
compression standards, e^, MPEG-1, MPEG-2, ITU-T H.261 and others, the 
range of resolutions in a specific implementation is preferably specified and is 
5 governed by the capabilities of the particular camera (14 or 44) and subsampler 

(1 8 or 48) used. Alternatively, it may be useful to limit the numbers of such 
resolutions to a small set, assigning each a unique code. Temporal resolution 
may likewise be designated within a discrete range of values, albeit there may 
be constraints to multiples of the minimum frame period, e. g. T l/30th of a 

10 second. Quantization, typically within the purview of the equipment 

manufacturers, may likewise be selected within a range, e.g. T supplied by the 
particular manufacturer. In this manner, it should further be understood that the 
encoder 50, upon receipt of a given codeword such as "q", may set a plurality 
of internal parameters accordingly. Similarly, various internal parameters of 

15 buffer 52 may also be set. 

Further, the particular viewer request transmitted through the 
backchannel 72 may be defined to be valid in a number of different ways, 
including: 

(A) forever, until a new request is given; 
20 (B) for a limited time period, namely x seconds, where x is pre- 

defined; 

(C) for a limited number of n images, where n is predefined, 

(D) as long as a transmitted codeword "x" indicates; and 

(E) as many images as a transmitted codeword "n" indicates. 

25 For example, a user may request a very high spatial resolution image, which may 

constitute a high resolution still image, by pressing button 74 (or button 76 on 
remote 78), which causes the translator 70 to generate a specific codeword, e.g. . 
"s" for still, which would be defined to be valid for a single image (n=l for 
option C above). The entire command transmitted across backchannel 72 would 

30 then be "sCl" Also, "g3A" may indicate a quantization level 3 from now on, 

and "tlB5" would similarly be temporal resolution level 1 for 5 seconds. 
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In view of the above description, it should be understood that a wide 
variety of non-normative commands may be utilized by a viewer to customize 
video images for various usages. 

Further details of the still picture management aspects of the video 
5 system 40 configuration of the present invention may be found in Applicants' 

co-pending patent application, entitled "Method and Apparatus for Still Picture 
Transmission and Display", filed concurrently herewith and incorporated herein 
by reference. 

It should be understood that the aforedescribed principles of the 

10 present invention may also be applied in other contexts, such as video 

conferencing, whereby a viewer may adapt the received signal to suit their 
needs. For example, if identification of individuals or objects is important, then 
an increase in spatial resolution and finer quantization (less distortion) is in order 
at the expense of temporal resolution. If image continuity is important, then 

15 sacrifices may be made in spatial resolution and distortion to achieve temporal 

stability. Other variations, such as taking a still image of a remote participant 
or fine-tuning the received image for further applications, should be considered 
to be within the purview of the present description. 

It should be understood that sending device 42 may be resident within 

20 camera 44 and receiving device 58 may be resident within the display device 66, 

such as in conventional video systems. It should, nonetheless, be understood 
that both the sending 42 and receiving 58 devices may instead be incorporated 
within multifunction terminals, e.g. . as a video-capable mobile 
transmitter/receiver, implemented in software (with the requisite hardware 

25 components such as the interface 69) within a general computer, implemented 

as separate equipment boxes with input and output connectors, or included (at 
least at the receiving end) as part of a value-added-service network, i.e. . a 
network shared resource. 

It should further be understood that control unit 54 shown in FIGURE 

30 2 may also control operations of the camera 44 through a connection thereto 
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(not shown). In this manner, the camera zoom, pan, tilt, iris and other camera 
functions may be user controlled because of backchannel 72. 

The previous description is of preferred embodiments for implementing 
the invention, and the scope of the invention should not necessarily be limited 
5 by this description. The scope of the present invention is instead defined by the 

following claims. 
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WHAT IS CLAIMED IS: 

1 . An imaging system for receiving, processing and displaying a 
multiplicity of images, said imaging system comprising: 

a sending means, connected to a camera means, for receiving said 
5 multiplicity of images therefrom; 

a processing means, within said sending means, for processing said 
multiplicity of images and generating a digital signal corresponding to said 
multiplicity of images, said digital signal comprising a plurality of operating 
parameters; 

10 a receiving means, connected to said sending means and said 

processing means via a forward channel, for receiving said digital signal for said 
multiplicity of images; 

a decoding means, within said receiving means, for decoding said 
digital signal and reconstituting said multiplicity of images therefrom pursuant 
15 to said plurality of operating parameters; 

a display for displaying said multiplicity of images from said receiving 
means; and 

an interface, connected to said receiving means, said interface, in 
response to a command to modify at least one of said operating parameters, 
20 forwarding said command to said processing means within said sending means 

via a backchannel thereto, whereby said processing means modifies said digital 
signal. 
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